Skip to content

Add parquet benchmark - #23596

Open
y2kiran wants to merge 1 commit into
NVIDIA:mainfrom
y2kiran:ykiran-dict-transcode-bench
Open

Add parquet benchmark#23596
y2kiran wants to merge 1 commit into
NVIDIA:mainfrom
y2kiran:ykiran-dict-transcode-bench

Conversation

@y2kiran

@y2kiran y2kiran commented Aug 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Description

This PR adds benchmarks to test the newly added output_dict_columns options for the Parquet reader, which was introduced in this PR

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@y2kiran
y2kiran requested review from a team as code owners August 7, 2026 22:08
@github-actions github-actions Bot added libcudf Affects libcudf (C++/CUDA) code. CMake CMake build issue labels Aug 7, 2026
@coderabbitai

coderabbitai Bot commented Aug 7, 2026 •
edited
Loading

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Summary by CodeRabbit

  • New Features
    • Added a Parquet dictionary-transcoding benchmark.
    • Compares string materialization, dictionary encoding, and direct dictionary transcoding.
    • Reports timing, memory usage, throughput, file size, and direct-transcoding speedups across configurable data and I/O scenarios.

Walkthrough

Changes

Parquet dictionary benchmark

Layer / File(s) Summary
Benchmark modes and result reporting
cpp/benchmarks/io/parquet/parquet_reader_dict.cpp		 Defines three read modes, dictionary-width estimation, run configuration, timing collection, and comparison output.
Fixture generation and mode execution
cpp/benchmarks/io/parquet/parquet_reader_dict.cpp		 Generates configurable dictionary-encoded Parquet data and runs string, dictionary-encoded, and direct-transcoding reads.
Validation, metrics, and target wiring
cpp/benchmarks/io/parquet/parquet_reader_dict.cpp, cpp/benchmarks/CMakeLists.txt		 Validates output types, skips plain-encoded direct-transcode cases, records performance metrics, configures benchmark axes, and adds the source to the NvBench target.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related PRs

  • rapidsai/cudf#22532: Introduces the same Parquet dictionary reader benchmark implementation and CMake wiring.

Suggested labels: improvement

Suggested reviewers: pointkernel, mhaseeb123, pmattione-nvidia, davidwendt, devavret

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title accurately identifies the main change as adding a Parquet benchmark, although it does not specify the reader dictionary-transcoding focus.
Description check ✅ Passed The description directly explains that the pull request adds benchmarks for the Parquet reader's output_dict_columns options.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mattgara mattgara Awaiting requested review from mattgara mattgara was automatically assigned from rapidsai/cudf-cpp-codeowners
@misiugodfrey misiugodfrey Awaiting requested review from misiugodfrey misiugodfrey was automatically assigned from rapidsai/cudf-cpp-codeowners

At least 1 approving review is required to merge this pull request.

Assignees

@y2kiran y2kiran

Labels

CMake CMake build issue libcudf Affects libcudf (C++/CUDA) code.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@y2kiran